In the digital image processing of animation data photographed by a video camera, noise is also amplified when the brightness is enhanced by gain adjustment to dark data or when the detail of the animation data is highlighted by image processing such as a picture improvement filter. Therefore, some parts cannot be clearly seen due to noise interference even if the brightness is enhanced or detailed data is highlighted.
In general, this noise is random, so the noise can be removed by taking an average over a plurality of animation frames. A commercial video camera comprises a slow shutter mode as well as a gain adjustment for enhancing the brightness of dark data. The camera has problems set out below even if the brightness is enhanced and the noises are muted.
Problem 1
The shutter speed is slow and it cannot secure a reasonable frame rate. For example, the camera cannot clearly capture a moving target with a shutter speed of only 5 shutter closures per second.
Problem 2
The accumulation of a plurality of frames and a prolonged open shutter time causes blurring and a moving target is not recognized.
Problem 3
Under a situation of low/narrow dynamic range with bright conditions (such as fog), the camera cannot be used due to over exposure.
Problem 4
Even in a dark picture, over exposure in a bright part of the picture causes highlight clipping.
Problem 5
Thermal noise (dark noise) is conspicuous, it is amplified by the act of exposing the noise to too much light.
To solve these problems, an image processing system is considered that adds image data to a frame buffer at the same ratio to perform a processing of noise reduction and sensitization (for enhancing the brightness of dark data) simultaneously.
When a dark image is processed by the above methods, one plane of a frame buffer for image accumulation is prepared as an accumulation buffer, ratio R and pixel data of an image frame are multiplied and this multiplied value is added to the accumulation buffer and this process is repeated. In this method, by continuing to add the value of the image data in ratio R to a frame buffer, the brightness of the image becomes the infinite geometric series of ratio R. The sum of infinite geometric series is equal to A/(1−R). Therefore, when N is the objective magnification of brightness, R is equal to (N−1)/N. (First term is A, A is equal to 1 when N is greater than 1)
If the magnification of the brightness is greater than 1, overflow is caused. Therefore, when it is required to calculate an image of 8 bits for in magnification N, the bit of the accumulation buffer has to be 8+base-2 of logarithm. (8+log 2) As a result of the above, objective brightness can be obtained some frames later after accumulation.
For example, a value of objective magnification of the brightness is 8, R is ⅞. Therefore, the content of the frame buffer is ⅞, and a new frame is added to the frame buffer, and it is stored in the accumulation buffer. For example, after processing 30 frames using the above method, the brightness of the image becomes the value multiplied by 8 in comparison with the original image.
If an object is a bright image, it is set so that the sum of the first term A and the proportional value R become 1 (A+R=1). When R is greater than 0 and less than 1 (0<R<1), the sum converges to 1. This means that the brightness is not changed.
Using the above method, noise can be removed without a change in the brightness since a plurality of the image is accumulated. For example, in the situation with that R is equal to one over four and A(1) is equal to three over four (R=¼, A(1)=¾), frame buffer is set at three over four (¾). Also, a new frame with a size of one over four is added to the frame buffer and it is stored in accumulated frame buffer. Repeating this process makes a transformation of the size of data 25%→18%→14%→10%→8%→6%. Also, each value is added to the frame buffer. Therefore, after having repeated 12 frames, the brightness obtained for the accumulated image is the same as the brightness of the original image.
Applying the above method, removal of noise and sensitization handling of data by the accumulation method can be carried out simultaneously, and problems 1, 3 and 5 can be solved.
Regarding problem 1, since the shutter speed of the frame using the method is the same as the traditional speed and the image update is carried out by each frame unit, the problem regarding the chasing of the moving object is not generated. Also, the ratio of the last frame in the image is the largest, thus the picture can clearly be seen when the plurality of frame buffers are added.
Regarding problem 2, since accumulation is carried out by each frame unit, a blurring is not caused. However, an afterimage exists, and a moving object seems to leave a trail.
Regarding problem 3, because magnification of the brightness can accumulate with 1, no problem occurs with a bright image. Also, it is not necessary to increase or decrease a frame buffer because the stockpile can be adjusted by using a parameter. Regarding problem 4, if magnification of the brightness becomes greater than 1, the problem remains of a white area at the point of overflow by accumulation. Regarding problem 5, because the shutter speed is the same or slower than the traditional speed of frame, the problem of heat noise does not occur.
Therefore, in the method in which one piece of accumulation buffer is repeated so that the same ratio is multiplied to a new frame and added to the accumulation buffer is very effective in noise removal and sensitization. However, there still remain the problems of white clipping when magnification of the brightness is raised and of a moving object having an afterimage.
An image processing device for still images using the following means is disclosed in patent document 1. Means for deciding the quantity of revision; determine the quantity of revision of brightness of the image data and the quantity of contrast revision of the image data by using a brightness reference value and contrast reference value which is the base for adjustment of the brightness of the picture. Means for correcting a quantity of revision of brightness; this means decrease a quantity of brightness amendment as a quantity of an exposure amendment increase. Means for correcting a quantity of contrast amendment; when the exposure amendment quantity is positive amendment, this means reduce the quantity of contrast amendment as the exposure amendment quantity increases. Picture conditioning means; this means adjust the brightness of the image data by applying quantity of revision of the corrected brightness and adjust the contrast of the image data by applying the quantity of revision of the corrected contrast.
Also, a data processor is disclosed in Prior art 2, the data processor processes three-dimensional data by a neural network, the neural network processes the three-dimensional data as quaternion, the data processor processes photography data of night scope 50 including a very small amount of RGB ingredient, and it can obtain a color image equivalent to what was photographed in the daytime